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PREFACE 


This  report  summarizes  the  findings  of  a  three-year  investigation  of  cognitive 
processes  in  planning  and  control,  conducted  for  the  Office  of  Naval  Research  under 
Contract  No.  N00014-78-C-0039. 

The  project  entailed  developing  a  cognitive  model  of  the  planning  process, 
formalizing  the  model  as  a  computer  simulation,  and  collecting  a  variety  of  empiri¬ 
cal  and  computational  data.  The  findings  should  interest  practitioners  and  re¬ 
searchers  concerned  with  planning  performance  and  the  development  of  planning 
aids.  More  detailed  discussions  of  project  subtasks  can  be  found  in  the  following 
Rand  publications: 

Cognitive  Processes  in  Planning,  by  Barbara  Hayes-Roth  and  Frederick 
Hayes-Roth,  R-2366-ONR,  December  1978. 

Decisionmaking  During  the  Planning  Process,  by  Barbara  Hayes-Roth  and 
Perry  W.  Thorndyke,  N-1213-ONR,  October  1980. 

Flexibility  in  Executive  Strategies,  by  Barbara  Hayes-Roth,  N-1170-ONR. 
September  1980. 

Individual  Differences  in  Planning  Processes,  by  Sarah  E.  Goldin  and  Bar¬ 
bara  Hayes-Roth,  N-1488-ONR,  June  1980. 

Modeling  Planning  as  an  Incremental,  Opportunistic  Process,  by  Barbara 
Hayes-Roth,  Frederick  Hayes-Roth,  Stanley  J.  Rosenschein.  and  Stephanie 
Cammarata,  N-1178-ONR,  June  1979. 

Estimation  of  Time  Requirements  During  Planning:  Interactions  Between 
Motivation  and  Cognition,  by  Barbara  Hayes-Roth.  N-1581-ONR.  Novem¬ 
ber  1980. 


SUMMARY 


Planning  and  control  are  primary  components  of  effective  management.  Plan¬ 
ning  refers  to  formulating  an  intended  course  of  action  aimed  at  achieving  a  goal; 
control  refers  to  monitoring  and  guiding  the  plan  to  a  successful  conclusion.  This 
report  summarizes  the  results  of  a  three-year  research  effort  investigating  the 
cognitive  processes  underlying  planning  and  control. 

The  project  focused  on  problems  analogous  to  the  naval  tactical  planning  prob¬ 
lem:  How  should  the  decisionmaker  move  force  units  (ships)  from  their  current 
locations  to  particular  task-force  objectives?  For  pragmatic  reasons,  we  did  not  use 
real  tactical  planning  problems  in  the  research.  However,  our  research  tasks  pre¬ 
served  the  basic  features  of  the  tactical  planning  problem:  The  planners  had  to 
consider  the  special  characteristics  of  individual  units,  the  locations  of  and  routes 
for  approaching  target  destinations,  the  goals  to  be  achieved  at  target  destinations, 
and  a*ner  risks  and  constraints. 

Our  first  subtask  was  to  develop  a  cognitive  model  of  the  planning  process.  This 
model,  which  we  call  the  opportunistic  planning  model,  views  planning  as  the 
cooperative  efforts  of  many  independent  plan  specialists,  or  condition-action  rules. 
Each  specialist  makes  tentative  decisions  for  incorporation  into  the  developing  plan 
and  posts  them  in  a  common  data  structure,  the  blackboard.  Each  specialist  can 
examine  prior  decisions  posted  on  the  blackboard  and  consider  them,  along  with 
ito  own  "knowledge,”  in  generating  new  decisions.  The  blackboard  is  partitioned 
into  several  planes,  representing  different  conceptual  categories  of  decisions.  These 
include  (a)  metaplan  decisions — what  general  approach  to  take  to  the  problem;  (b) 
plan  decisions — what  actions  to  take;  (c)  plan  abstraction  decisions — what  kinds  of 
actions  are  desirable;  kl)  world  knowledge  decisions — what  information  motivates 
particular  plan  decisions;  and  (e)  executive  decisions — how  to  organize  the  planning 
process  itself.  Each  of  these  planes  is  further  partitioned  into  several  levels  of 
abstraction.  The  blackboard  structure  restricts  the  amount  of  information  each 
specialist  must  consider  in  order  to  generate  a  decision.  It  also  provides  a  conceptu¬ 
al  framework  for  understanding  and  analyzing  the  planning  process.  This  process 
proceeds  through  a  series  of  cycles  during  which  individual  specialists  inspect 
information  on  the  blackboard  and  post  new  decisions.  The  scheduling  of  individual 
specialists  is  controlled  by  executive  decisions  based  on  focus  of  attention  and 
recency  of  invocation.  The  process  ordinarily  continues  until  the  planner  has  inte¬ 
grated  mutually  consistent  decisions  into  a  satisfactory  plan. 

We  implemented  a  computer  simulation  of  the  opportunistic  model  in  the  pro¬ 
gramming  language  INTERLISP.  The  simulation  contains  an  internal  representa¬ 
tion  of  the  blackboard  structure  and  about  forty  specialists.  Its  performance  is  quite 
similar  to  that  of  a  human  planner.  The  simulation  and  the  human  planner  produce 
plans  with  similar  strengths  and  weaknesses.  In  addition,  the  "protocols”  they 
produce  while  planning  contain  many  of  the  same  intermediate  decisions,  as  well 
as  many  common  general  characteristics. 

Empirical  research  performed  in  this  study  encompassed  a  variety  cf  tasks: 


v 


VI 


1.  Protocol  analyses  confirmed  the  basic  assumptions  of  the  opportunistic 
planning  model:  (a)  that  people  make  decisions  representing  each  of  the 
postulated  levels  of  abstraction  on  all  five  planes  of  the  planning  black¬ 
board;  and  (b)  that  people  generate  successive  decisions  opportunistically, 
rather  than  by  working  systematically  through  the  planning  space. 

2.  We  obtained  more  rigorous  support  for  these  assumptions  in  experimental 
investigations  of  subjects’  judgments  about  the  similarity  of  individual 
planning  decisions  and  investigations  of  the  influence  of  early  planning 
decisions  on  subsequent  decisions. 

3.  Another  set  of  experiments  demonstrated  subjects’  abilities  to  adopt  alter¬ 
native  general  planning  strategies.  Subjects  adopted  particular  strategies 
in  response  to  explicit  instruction  and  by  transferring  strategies  adopted 
on  previous,  similar  planning  tasks. 

4.  A  series  of  detailed  protocol  analyses  revealed  three  categories  of  individ¬ 
ual  differences  in  planning  behavior:  Good  planners  made  decisions  in  all 
categories  specified  on  the  planning  blackboard,  whereas  poor  planners 
operated  in  a  more  restricted  region  of  the  blackboard;  good  planners 
showed  greater  attentiona!  flexibility  than  poor  planners:  and  good  plan¬ 
ners  had  many  more  planning  specialists  than  poor  planners.  Based  on 
these  analyses,  we  developed  a  protocol  for  effective  planning. 

5.  Correlational  analyses  suggested  that  several  scenario  characteristics 
might  influence  planning  performance.  These  included  the  number  of 
goals  under  consideration,  the  time  available  for  achieving  goals,  and  the 
difficulty  of  achieving  all  goals  in  the  available  time. 

6.  A  series  of  studies  on  time  estimation  during  planning  revealed  that  sub¬ 
jects  habitually  underestimate  the  time  required  for  planned  tasks.  Two 
factors  seem  to  contribute  to  this  tendency.  The  cognitive  factor  is  sub¬ 
jects'  inclination  to  plan  at  high  levels  of  abstraction,  failing  to  enumerate 
all  time-coi.-suming  elements  of  planned  activities.  The  motivational  factor 
is  subjects’  tendency  to  generate  low  time  estimates  in  response  to  a  desire 
to  accomplish  all  or  most  of  the  goals  under  consideration. 

7.  We  developed  an  automatic  plan  evaluator  to  provide  feedback  to  subjects 
on  their  planning  performance.  By  interacting  with  the  evaluator  on  sev¬ 
eral  trials,  subjects  learned  about  relevant  environmental  contingencies 
and  the  criteria  used  to  evaluate  their  plans.  We  then  measured  the  qual¬ 
ity  of  subjects’  plans  in  terms  of  their  asymptotic  performance  following 
these  trials. 

Based  on  the  results  summarized  above,  we  propose  a  set  of  principles  for 
improving  planning  performance: 

•  The  criteria  for  selecting  planners  should  be  a  large-capacity  working 
memory,  attentional  flexibility,  and  strategic  flexibility. 

•  Planners  should  be  taught  the  importance  of  abstract  plans,  world  knowl¬ 
edge,  and  metacognition. 

•  Planners  should  also  be  taught  domain-specific  planning  heuristics,  the 
value  of  controlled  opportunism,  general  planning  strategies,  and  cogni¬ 
tive  strategies  for  resource  estimation. 


•  Three  training  methods  should  be  used:  explicit  instruction,  induction  of 
illustrative  experiences,  and  illustration  of  effective  planning  by  means  of 
a  computer  simulation. 

•  Three  planning  a:ds  should  be  employed  :  a  resource  estimator,  a  plan 
evaluator,  and  a  planning  collaborator. 

While  these  suggestions  must  be  regarded  as  preliminary,  they  provide  a  detailed 
framework  for  future  research. 
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I.  INTRODUCTION 


Planning  and  control  constitute  two  primary  components  of  effective  manage¬ 
ment.  Planning  refers  to  formulating  an  intended  course  of  action  aimed  at  achiev¬ 
ing  some  goal:  control  refers  to  monitoring  and  guiding  the  plan  to  a  satisfactory 
conclusion. 

Because  planning  and  control  processes  are  ubiquitous,  a  good  modei  of  these 
processes  should  apply  to  the  performance  of  many  tasks.  The  research  project 
described  in  this  report  focused  on  problems  similar  to  the  naval  tactical  planning 
problem:  How  should  the  decisionmaker  move  force  units  (ships)  from  their  current 
locations  to  particular  task-force  objectives?  In  approaching  such  a  problem,  the 
decisionmaker  must  consider  the  special  characteristics  of  individual  units  (e.g.. 
different  drafts  or  detection  ranges-,  the  locations  of  and  routes  for  approaching 
tr  >-get  destinations,  the  goals  to  be  achieved  at  target  destinations,  and  additional 
risks  and  constraints. 

Currently,  there  are  no  formal  methods  for  solving  these  problems.  Because  of 
their  computational  complexity,  tactical  planning  problems  are  not  amenable  to 
either  operations  research  procedures  or  computational  algorithms.  People  appar¬ 
ently  solve  these  problems  by  attending  selectively  to  critical  problem  components 
and  applying  heuristic  methods.  The  goal  of  the  research  reported  here  was  to 
improve  our  understanding  of  the  cognitive  processes  people  bring  to  bear  on 
tactical  planning  problems  and  to  infer  principles  for  the  improvement  of  planning 
performance.  This  report  summarizes  our  findings  and  presents  a  set  of  prelimi¬ 
nary  principles. 

The  remainder  of  the  report  is  organized  as  follows:  Section  II  describes  the 
task  used  in  our  research.  Section  III  characterizes  the  cognitive  model  developed 
to  describe  and  explain  human  planning  behavior  and  to  guide  the  subsequent 
research.  Section  IV  describes  a  compu’^r  simulation  of  the  planning  model  and 
compares  its  performance  to  the  performance  of  human  planners.  Section  V  sum¬ 
marizes  a  variety  of  empirical  research  on  human  planning  performance.  Section 
VI  presents  a  set  of  preliminary  principles  for  the  improvement  of  p!anning  perfor¬ 
mance. 
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II.  THE  RESEARCH  TASK 


For  pragmatic  reasons,  we  did  not  use  real  tactical  planning  problems  in  our 
research.  The  empirical  studies  required  large  numbers  of  readily  available  sub¬ 
jects,  and  it  would  have  been  very  difficult  and  expensive  to  use  only  subjects  with 
prior  knowledge  of  and  expertise  in  tactical  planning.  Therefore,  the  research  used 
an  analogue  of  the  tactical  planning  problem — an  errand-planning  task. 

For  this  task,  we  presented  subject  with  a  scenario  that  specified  a  starting 
time  and  location,  an  ending  time  and  fi...u  destination,  a  list  of  intermediate  target 
destinations  and  associated  goals,  and  certain  additional  constraints.  Subjects  then 
formulated  plans  for  achieving  a  subset  of  the  requested  goals  under  the  con¬ 
straints  specified  in  the  scenario.  A  sample  scenario  follows: 

You  have  just  finished  working  out  at  the  health  club.  It  is  11:00  and  you 
can  plan  the  rest  of  your  day  as  you  like.  However,  you  must  pick  up  your 
car  from  the  Maple  Street  parking  garage  by  5:30  and  then  head  home. 
You’d  also  like  to  see  a  movie  today,  if  possible.  Show  times  at  both  movie 
theaters  are  1:00,  3:00,  and  5:00.  Both  movies  are  on  your  "must  see”  list, 
but  go  to  whichever  one  most  conveniently  fits  into  your  plan.  Your  other 
errands  are  as  follows: 

•  Pick  up  medicine  for  your  dog  at  the  vet. 

•  Buy  a  fan  belt  for  your  refrigerator  at  the  appliance  store. 

•  Check  out  two  of  the  three  luxury  apartments. 

•  Meet  a  friend  for  lunch  at  one  of  the  restaurants. 

•  Buy  a  toy  for  your  dog  at  the  pet  store. 

.  Pick  up  your  watch  at  the  watch  repair. 

•  Special-order  a  book  at  the  bookstore. 

•  Buy  fresh  vegetables  at  the  grocery. 

•  Buy  a  gardening  magazine  at  the  newsstand. 

•  Go  to  the  florist  to  send  flowers  to  a  friend  in  the  hospital. 

Figure  1  shows  the  map  subjects  used  in  forming  their  plans. 

Like  the  sample  scenario,  most  of  the  scenarios  we  used  did  not  provide  enough 
time  to  accomplish  all  of  the  intermediate  goals.  Therefore,  the  subjects’  task 
required  them  to  (a)  decide  which  goals  to  achieve,  (b)  decide  the  order  in  which 
to  achieve  individual  goals,  (c)  plan  routes  between  successive  destinations,  and  (d) 
decide  how  much  time  to  allocate  for  individual  goals  and  for  traveling  between 
destinations. 

The  errand-planning  task  is  very  similar  to  the  tactical  planning  task:  How 
should  the  decisionmaker  move  units  (people)  from  their  current  locations  to  par¬ 
ticular  task  objectives?  In  approaching  such  a  problem,  the  decisionmaker  must 
consider  the  special  characteristics  of  individual  units  (e.g.,  mode  of  transportation, 
ability  to  carry  burdens),  the  locations  of  and  routes  for  approaching  target  destina¬ 
tions,  the  goals  to  be  achieved  at  target  destinations,  and  additional  risks  and 
constraints. 
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III.  A  COGNITIVE  MODEL  OF  THE  PLANNING  PROCESS 


The  first  project  subtask  was  to  develop  a  model  of  the  cognitive  processes 
people  use  during  planning.  This  model,  the  opportunistic  planning  model,  views 
planning  as  the  cooperative  effort  of  many  independent  plan  specialists.  Specialists 
may  be  thought  of  as  mental  rules  or  programs.  Each  specialist  makes  tentative 
decisions  for  incorporation  into  the  developing  plan.  Different  specialists  have 
different  types  of  planning  knowledge  and  hence  inf.aence  different  aspects  of  the 
plan.  For  example,  some  specialists  suggest  high-level,  abstract  additions  to  the 
plan,  while  others  suggest  detailed  sequences  of  specific  actions. 

In  the  opportunistic  planning  model,  specialists  record  their  decisions  in  a 
common  data  structure,  the  blackboard.  Each  specialist  can  examine  prior  deci¬ 
sions  posted  on  the  blackboard,  transform  or  combine  that  information  with  its  own 
"knowledge,”  and  generate  new  decisions.  The  blackboard  is  partitioned  into  sev¬ 
eral  planes  corresponding  to  different  conceptual  categories  of  decisions.  Each 
plane  is  further  partitioned  into  several  levels  of  abstraction.  This  blackboard 
structure  restricts  the  amount  of  information  each  specialist  must  consider  in  order 
to  generate  a  decision.  It  also  provides  a  conceptual  framework  for  understanding 
and  analyzing  planning  processes.  The  remainder  of  this  section  discusses  special¬ 
ists,  the  blackboard  structure,  and  control  of  the  planning  processes  in  more  detail. 
Finally,  it  contrasts  the  proposed  model  with  previous  conceptions  of  the  planning 
process. 


SPECIALISTS 

Specialists  embody  knowledge  about  different  kinds  of  planning  decisions  and 
the  situations  in  which  each  kind  is  appropriate.  The  model  operationalizes  special¬ 
ists  as  condition-action  rules.  For  example,  one  specialist  might  state 

IF  there  is  a  requested  errand  near  the  current  location, 

THEN  record  a  decision  to  perform  that  errand  next. 

The  condition  component  describes  the  circumstances  under  which  the  specialist 
can  make  a  decision.  The  condition  usually  requires  specific  types  of  information 
to  have  been  posted  on  the  blackboard  by  other  specialists,  as  well  as  the  satisfac¬ 
tion  of  other,  arbitrarily  complex  criteria.  When  the  condition  of  a  specialist  has 
been  satisfied,  we  say  that  the  specialist  has  been  invoked.  The  action  component 
specifies  the  specialist’s  decisionmaking  behavior.  The  action  might  post  a  new 
decision  on  the  blackboard  or  modify  a  previously  posted  decision.  Thus,  specialists 
represent  a  set  of  heuristics  for  generating  or  modifying  plan  decisions  based  on 
recognized  patterns  of  previous  decisions. 


BLACKBOARD  STRUCTURE 

As  mentioned  above,  the  blackboard  contains  five  conceptual  planes:  plan,  plan 
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abstraction ,  world  knowledge,  executive,  and  metaplan.  Each  of  these  partitions 
records  a  different  kind  of  planning  knowledge.  The  blackboard  structure  is  shown 
in  Fig.  2. 

Plan,  plan  abstraction,  and  world  knowledge  decisions  determine  features  of 
the  developing  plan.  Decisions  on  the  plan  plane  represent  actions  that  the  planner 
intends  to  take  in  the  world.  For  example,  in  the  errand-planning  task,  the  planner 
might  decide  to  go  to  the  florist  next  or  to  travel  down  Jackson  Avenue  to  get  to 
the  bank.  Decisions  on  the  plan  abstraction  plane  characterize  desired  attributes 
of  potential  plan  decisions,  indicating  the  kinds  of  actions  that  the  planner  would 
like  to  take,  without  specifying  the  actions  themselves.  For  example,  the  planner 
might  decide  to  go  to  the  closest  errand  next  (without  specifying  the  identity  of  that 
errand)  or  to  organize  the  plan  around  spatial  clusters  of  errands  (without  specify¬ 
ing  the  contents  of  those  clusters).  Decisions  on  the  world  knowledge  plane  record 
observations  and  computations  regarding  relationships  in  the  task  environment 
that  might  bear  cn  the  final  plan.  For  example,  a  world  knowledge  decision  mi^ht 
encode  the  fact  that  the  florist  is  close  to  the  current  location,  or  that  the  bank,  the 
shoe  store,  and  the  movie  theater  are  clustered  in  the  same  neighborhood. 

In  contrast  to  these  three  planes,  executive  decisions  determine  the  allocation 
of  cognitive  resources  during  the  planning  process:  what  kinds  of  decisions  to 
generate  first,  what  aspect  of  the  plan  to  develop  next,  what  specialist  to  bring  to 
bear  at  a  given  point  in  the  planning  process.  For  example,  the  planner  might 
decide  to  determine  the  best  order  for  the  errands  before  finding  a  route  among 
them. 

The  metaplan  plane  contains  more  general  decisions  about  how  to  approach  the 
planning  problem.  Metaplan  decisions  reflect  the  planner’s  understanding  of  the 


Fig.  2— Blackboard  structure  in  the  opportunistic  planning  model 
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problem,  the  methods  he  or  she  intends  to  apply  to  it,  and  the  criteria  he  or  she  will 
use  to  generate  and  evaluate  prospective  plans.  For  example,  the  planner  might 
decide  to  maximize  plan  efficiency  at  the  expense  of  certain  types  of  subgoals. 

As  mentioned  above,  the  model  further  partitions  each  plane  into  several  levels 
of  abstraction.  The  plan  abstraction,  world  knowledge,  and  plan  planes  each  have 
four  levels  of  abstraction  that  form  a  potential  hierarchy,  with  decisions  at  each 
level  specifying  a  more  refined  plan  than  those  at  the  next  higher  level.  Decisions 
at  the  highest  level  determine  a  plan’s  outcomes:  which  errands  will  be  accom¬ 
plished  when  the  plan  is  executed.  For  example,  outcome  decisions  on  the  plan 
abstraction,  world  knowledge,  and  plan  planes  might  be 

a.  I’ll  do  all  the  most  important  errands. 

b.  The  drugstore  and  the  grocery  are  the  most  important  errands. 

c.  I’ll  definitely  go  to  the  drugstore  and  the  grocery. 

Decisions  at  the  next  lower  level  determine  the  plan’s  design,  the  overall  spa- 
tial-t'  .nporal  organization  of  planned  activities.  For  example,  decisions  on  the 
three  planes  at  this  level  might  be 

a.  I’ll  organize  the  plan  around  spatial  clusters  of  errands. 

b.  There  is  a  cluster  of  errands  in  the  southwest  corner. 

c.  I’ll  head  toward  the'  southwest  cluster,  doing  errands  on  the  way. 

Decisions  at  the  next  lower  level  determine  the  plan’s  procedures,  that  is,  the 
ordering  of  individual  errands.  For  example,  three  such  decisions  might  be 

a.  I’ll  do  the  closest  errand  next. 

b.  The  florist  is  closest. 

c.  I’ll  go  to  the  florist  next. 

Finally,  decisions  at  the  lowest  level  of  abstraction  determine  the  plan’s  oper¬ 
ations,  the  details  of  performing  specific  errands  or  traveling  from  one  to  another. 
For  example,  three  such  decisions  might  be 

a.  I’ll  take  the  shortest  route  to  the  next  errand. 

b.  Belmont  Street  is  the  shortest  route  between  the  parking  lot  and  the 
florist. 

c.  I’ll  travel  via  Belmont  Street  to  get  to  the  florist. 

The  executive  plane  has  three  levels  of  abstraction.  At  the  highest  level,  priority 
decisions  establish  principles  for  allocating  cognitive  resources  during  the  entire 
planning  process  (e.g.,  I’ll  decide  which  errands  to  do  before  deciding  when  to  do 
any  of  them).  At  the  next  lower  level,  focus  decisions  indicate  what  kind  of  decisions 
to  make  at  a  particular  point  in  the  planning  process  (e.g.,  Now  I’ll  figure  out  the 
best  route  from  the  grocery  to  the  drugstore).  Finally,  scheduling  decisions  resolve 
any  remaining  conflicts  between  competing  invoked  specialists,  choosing  a  particu¬ 
lar  specialist  to  execute  its  action  next. 

The  four  levels  of  the  metaplan  plane,  problem  definition,  problem-solving 
model,  policies,  and  evaluation  criteria,  do  not  produce  a  neat  hierarchy.  However, 
they  emphasize  different  aspects  of  the  subject’s  approach  to  the  planning  problem: 
his  or  her  representation  of  the  task  and  its  goals,  resources  and  constraints  (prob¬ 
lem  definition),  the  general  strategy  the  planner  assumes  in  generating  a  solution 


(problem-solving  model),  a  set  of  global  constraints  and  desirable  features  for  the 
developing  plan  (policies),  and  a  set  of  dimensions  against  which  the  planner  can 
evaluate  tentative  plans  (evaluation  criteria). 


CONTROL  OF  THE  PLANNING  PROCESS 

The  planning  process  proceeds  through  a  series  of  cycles  during  which  special¬ 
ists  read  information  from  the  blackboard  and  take  sub?-  ^uent  action.  On  any 
cycle,  a  number  of  specialists  may  be  invoked — that  is,  their  conditions  may  have 
been  satisfied  by  the  appearance  of  some  prior  decision  on  the  blackboard.  An 
executive  decision  determines  which  of  the  invoked  specialists  executes  its  action, 
generating  a  new  decision  and  recording  it  on  the  blackboard.  This  new  decision 
will  invoke  additional  specialists,  beginning  a  new  cycle.  The  process  will  ordinarily 
continue  until  the  planner  has  integrated  mutually  consistent  decisions  into  a 
satisfactory  plan. 

The  executive  decisions  that  schedule  invoked  specialists  play  a  central  role  in 
the  performance  of  the  simulation.  These  decisions  reflect  previous  decisions  about 
area;  nf  the  blackboard  for  focusing  attention.  Other  things  being  equal,  specialists 
that  operate  in  these  areas  will  be  given  priority.  The  executive  also  considers 
recency  of  invocation.  Other  things  being  equal,  recently  invoked  specialists  will  be 
given  priority.  These  two  considerations,  focus  of  attention  and  recency  of  invoca¬ 
tion,  enable  the  executive  to  schedule  invoked  specialists  so  as  to  produce  a  variety 
of  intelligent  decision  sequences. 


CONTRASTS  BETWEEN  THE  OPPORTUNISTIC  PLANNING  MODEL 
AND  PREVIOUS  PLANNING  MODELS 

The  proposed  model  contrasts  with  previous  modeis  of  planning  as  a  systemat¬ 
ic,  "top-down”  process.  These  earlier  models  assume  that  a  planner  begins  by 
making  very  abstract  plans  which  guide  and  restrict  subsequent  development  of 
more  detailed  plans.  While  the  opportunistic  model  recognizes  the  important  role 
of  top-down  planning,  it  also  incorporates  "bottom-up”  planning  processes.  That  is, 
the  planner  sometimes  makes  very  detailed  plans  which  inspire  new  or  revised 
plans  at  higher  levels  of  abstraction. 

A  second  difference  concerns  the  relative  completeness  attributed  to  abstract 
plans.  The  earlier  models  assume  that  initial  abstract  plans  must  be  complete  and 
fully  integrated  at  seme  point  in  the  planning  process.  By  contrast,  the  opportunis¬ 
tic  model  assumes  that  planning  is  incremental  and  will  rarely  produce  complete 
plans  at  all  levels  of  abstraction.  It  assumes  that  planners  make  tentative  decisions, 
without  requiring  that  each  one  fit  into  a  current,  well-integrated  plan.  Alternative 
plans  and  subplans  can  develop  simultaneously,  being  integrated  into  a  final  com¬ 
plete  plan  at  the  planner’s  discretion. 

A  third  difference  concerns  the  presumed  structure  of  the  planning  process. 
Earlier  conceptions  treated  plans  as  hierarchical  structures,  while  the  opportunis¬ 
tic  model  treats  plans  as  more  general,  heterarchical  structures.  While  they  are 
well-structured,  the  many  categories  of  decisions  embodied  in  the  proposed  plan¬ 
ning  blackboard  do  not  fit  into  any  obvious  hierarchical  structure. 
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The  following  sections  present  a  variety  of  evidence  supporting  the  opportunis¬ 
tic  model.  They  do  not  present  a  detailed  analysis  of  the  contrasting  predictions  of 
the  opportunistic  model  and  the  top-down,  hierarchical  model.  However,  such 
analyses  can  be  found  in  the  technical  reports  noted  in  the  Preface  (p.  iii). 

In  addition  to  the  empirical  evidence,  we  might  speculate  on  the  relative  merits 
of  hierarchical  versus  opportunistic  planning.  On  one  hand,  the  orderly,  systematic 
nature  of  the  top-down  process  and  the  simplicity  of  its  hierarchical  structure  argue 
in  its  favor.  One  might  also  argue  that  a  top-down  process  would  minimize  memory 
load — the  planner  could  restrict  attention  to  a  single  area  of  the  hierarchy,  rather 
than  attending  intermittently  to  several  different  areas  of  the  planning  space. 

On  the  other  hand,  planning  in  tasks  fraught  with  complexity  and  uncertainty 
might  benefit  from  less  of  the  discipline  imposed  by  a  strictly  top-down  process.  For 
such  tasks,  general  problem-solving  methods  may  not  exist  or  may  be  computation¬ 
ally  intractable.  More  importantly,  a  multidirectional  process  might  produce  better 
plans.  Rather  than  restricting  consideration  to  the  obvious  refinements  of  a  fixed 
set  of  abstract  plans,  the  planner  can  develop  novel  high-level  plans  suggested  by 
low-level  observations  and  decisions.  Thus,  the  bottom-up  component  in  multidirec¬ 
tional  processing  provides  a  potentially  important  source  of  innovation  in  planning. 


IV.  COMPUTER  SIMULATION  OF  THE  OPPORTUNISTIC 

PLANNING  MODEL 


We  have  implemented  a  computer  simulation  of  the  planning  model  in  INTER¬ 
LISP.  The  simulation  contains  an  internal  representation  of  the  map  shown  in  Fig. 
i,  a  blackboard  structure  to  organize  planning  decisions,  and  about  forty  specialists. 
We  designed  the  specialists  to  model  knowledge  inferred  from  thinking-aloud 
protocols  produced  by  human  subjects  during  planning.  These  specialists  are  ab¬ 
stractions,  rather  than  literal  codings  of  the  observed  knowledge.  For  example,  the 
statement,  "Pll  go  to  the  drugstore  next  because  it’s  close,”  is  abstracted  as 

IF  an  errand  has  been  planned, 

THEN  plan  to  perform  the  closest  errand  to  the  planned  errand 
next. 

Thus,  the  simulation  is  not  finely  tuned  to  mode!  any  particular  protocol  or  subject. 
It  is  capable  of  producing  a  great  variety  of  decision  sequences  and  a  great  variety 
of  final  plans. 

We  can  evaluate  two  aspects  of  the  simulation’s  performance:  the  plans  it 
produces  and  the  process  by  which  it  produces  them.  We  discuss  each  of  these 
below. 

Figure  3  shows  the  plan  one  subject  produced  for  the  problem  discussed  above. 
Figure  4  shows  one  of  the  plans  produced  by  the  simulation.  The  two  plans  are  quite 
similar.  Both  plans  include  all  primary  errands  and  at  least  some  of  the  secondary 
errands.  While  the  human  subject  included  all  secondary  errands,  the  simulation 
included  only  one  very  convenient  secondary  errand.  Although  the  simulation  and 
the  subject  planned  different  routes,  both  routes  are  fairly  efficient,  though  clearly 
suboptimal.  Both  the  simulation  and  the  subject  planned  to  arrive  at  time-con¬ 
strained  destinations  (t.g.,  the  restaurant  and  the  movie)  at  reasonable  times.  The 
major  difference  between  the  two  plans  lies  in  their  relative  "realism.”  The  human 
subject’s  plan  is  quite  unrealistic — one  could  not  execute  the  complete  plan  in  the 
time  available  for  doing  so.  The  simulation’s  plan  is  somewhat  more  realistic, 
primarily  because  it  omits  many  of  the  secondary  errands. 

As  mentioned  earlier,  the  simulation  is  capable  of  producing  a  variety  of  plans. 
Other  "runs”  of  the  simulation  produced  plans  that  differed  more  markedly  from 
this  subject’s  plan.  If  we  experimented  further  with  the  simulation,  it  would  prob¬ 
ably  produce  plans  that  more  closely  resembled  the  subject’s  plan.  The  present 
exercise  simply  demonstrates  the  simulation’s  ability  to  produce  plans  that  are 
quite  similar  to  those  produced  by  human  planners.  This  is  an  important  sufficiency 
test  of  the  opportunistic  model. 

Like  the  human  subject,  the  simulation  produced  a  planning  protocol — the 
series  of  decisions  underlying  the  final  plan.  Figure  5  shows  an  excerpt  from  the 
subject’s  protocol,  and  Fig.  6  shows  a  corresponding  excerpt  from  the  simulation’s 
protocol — decisions  10-22.  (The  protocol  actually  produced  by  the  simulation  is  a 
series  of  decision  "nodes"  in  list  notation.  Figure  6  presents  the  protocol  in  standard 
English  for  clarity.) 
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produced 


1-4  Omitted 

*  5  All  right.  We  are  now  at  the  health  club. 

*  6  What  is  going  to  be  the  closest  one? 

*  7  The  appliance  store  is  a  few  blocks  away. 

*  8  The  medicine  for  the  dog  at  the  vet’s  office  isn’t 

too  far  away. 

*  9  Movie  theaters— let's  hold  off  on  that  for  a  little 

while. 

*10  Pick  up  the  watch.  That’s  all  the  way  across  town. 

*  1 1  Special-order  a  book  at  the  bookstore. 

*  12  Probably  it  would  be  best  if  we  headed  in  a 

southeasterly  direction.  Start  heading  this  way. 

*13  1  can  see  later  on  there  are  a  million  things  I  want 
to  do  in  that  part  of  town. 

Fig.  5— Excerpt  from  the  human  subject’s  protocol 


1-9  Omitted 

10  Work  forward  from  the  starting  location. 

*11  The  starting  location  is  the  health  club  which  is  on 
Belmont  Avenue. 

12  Begin  at  the  health  club. 

’  13  Go  to  the  closest  errand  next. 

*  14  The  vet  is  on  Oak  Street. 

*  15  The  appliance  store  is  on  Walnut  Street. 

16  The  restaurant  is  on  Washington  Avenue. 

*17  The  watch  repair  is  on  Cedar  Street. 

*  18  The  bookstore  is  on  Cedar  Street. 

19  The  grocery  is  on  Cedar  Street. 

*20  The  watch  repair,  the  bookstore,  and  the  grocery 
are  all  in  a  cluster  in  the  southeast  part  of  town. 

21  Organize  the  plan  around  spatial  clusters. 

*22  Proceed  from  the  Health  Club  toward  the 
southeast  cluster. 


Fig.  6.— Excerpt  from  the  simulation’s  protocol 


Decisions  10-22  of  the  simulation’s  protocol  correspond  quite  closely  to  items 
5-8  of  the  subject’s  protocol.  In  these  items,  the  subject  identified  the  health  club 
as  .1m.  starting  location,  decided  to  schedule  the  closest  errand  to  the  health  club 
next,  located  several  errands  on  the  map  in  his  search  for  the  closest  errand, 
detected  a  cluster  of  errands  in  the  southeast  part  of  town,  and,  accordingly,  de¬ 
cided  to  head  in  that  general  direction. 

Now  consider  the  simulation’s  protocol.  Decision  10  establishes  a  strategy  of 
working  forward  from  the  starting  location.  Decisions  11  and  12  identify  the  start¬ 
ing  location  as  the  health  club  and  establish  it  as  the  first  errand  in  the  procedure. 
Decision  13  establishes  a  strategy  of  going  to  the  closest  errand  next.  Decisions 
14-19  locate  individual  errands  on  the  map  in  a  search  for  the  closest  errand  to  the 
health  club.  Decision  20  detects  the  cluster  of  errands  in  the  southeast  part  of  town. 
Decision  21  suggests  organizing  an  overall  design  for  the  plan  around  the  spatial 
cluster,  and  decision  22  does  so 

In  addition  to  performing  essentially  the  same  functions  as  the  subject,  the 
simulation  made  many  of  the  decisions  explicitly  declared  in  the  subject’s  protocol. 
Common  decisions  are  preceded  by  an  asterisk  in  Figs.  5  and  6.  The  remainder  of 
the  simulation’s  protocol  does  not  always  mirror  the  subject’s  protocol  as  closely  as 
does  the  section  in  Fig.  6.  Because  the  simulation  can  produce  a  variety  of  decision 
sequences,  other  "runs”  would  produce  protocols  more  or  less  similar  to  the  sub¬ 
ject’s  protocol  However,  two  factors  limit  the  maximal  similarity  we  might  ob¬ 
serve. 

First,  the  simulation  does  not  contain  exactly  the  same  set  of  specialists  used 
by  the  subject.  Thus,  the  simulation  occasionally  uses  a  specialist  that  is  slightly 
different  from  the  one  the  subject  uses.  This  produces  differences  in  both  the 
protocols  and  the  resulting  plans.  For  example,  in  item  18  of  his  protocol,  the  subject 
decided  to  go  from  the  health  club  to  the  vet  on  the  way  to  the  southeast  part  of 
town.  At  the  same  point  in  its  protocol,  the  simulation  decided  to  go  from  the  health 
club  to  the  appliance  store  on  the  way  to  the  southeast  part  of  town.  Both  were 
trying  to  find  the  closest  errand  along  the  way,  but  they  used  slightly  different 
specialists,  and  as  a  consequence,  they  chose  different  errands. 

Second,  the  simulation’s  executive  is  incomplete.  On  some  cycles,  two  or  more 
invoked  specialists  are  equally  attractive  and  the  simulation  chooses  random!}' 
among  them.  Frequently,  this  random  choice  fails  to  select  the  specialist  the  subject 
used  at  that  point.  In  such  cases,  the  protocols  again  diverge. 

In  our  opinion,  it  would  be  unproductive  to  model  the  subject’s  performance  at 
a  level  of  detail  sufficient  to  counteract  the  effects  of  these  two  factors.  Therefore, 
we  look  for  the  same  genera!  features  in  the  two  protocols,  rather  than  exact 
replication.  Such  commonalities  are  readily  apparent.  Both  the  simulation  and  the 
subject  made  decisions  at  various  levels  of  abstraction  on  each  of  the  five  planes 
of  the  planning  blackboard.  Both  exhibited  many  coherent  decision  sequences  in 
which  each  decision  appeared  to  build  on  its  predecessors.  However,  both  also 
frequently  "jumped  about”  the  planning  blackboard,  rather  than  working  system¬ 
atically  along  any  particular  dimension.  In  particular,  the  simulation  and  the  sub¬ 
ject  both  occasionally  redirected  or  dramatically  changed  their  own  activity  in 
response  to  fortuitous  observations  or  computations  on  the  available  data  fi.e.,  the 
map).  Subsequent  runs  of  the  simulation  produced  protocols  that  differed  consider¬ 
ably  in  the  details  of  individual  decisions  but  exhibited  these  same  general  features. 
Again,  the  simulation’s  planning  behavior  establishes  the  sufficiency  of  the  oppor¬ 
tunistic  model  to  represent  human  planning  behavior. 


V.  EMPIRICAL  RESEARCH 


We  conducted  a  variety  of  empirical  research  tasks  to  test  the  assumptions  of 
the  opportunistic  planning  model,  explore  individual  differences  in  planning  perfor¬ 
mance,  and  characterize  general  properties  of  human  planning  performance.  These 
studies  and  their  results  are  summarized  briefly  below. 


PROTOCOL  ANALYSES 

Our  first  empirical  studies  were  informal  analyses  of  protocols  generated  by 
subjects  during  planning.  This  exercise  confirmed  the  basic  assumptions  of  the 
opportunistic  model:  (a)  that  people  make  decisions  representing  each  of  the  post¬ 
ulated  levels  of  abstraction  on  all  five  planes  of  the  planning  blackboard,  and  (b) 
that  people  generate  successive  decisions  opportunistically,  rather  than  by  working 
systematically  through  the  planning  space. 


TESTING  BASIC  ASSUMPTIONS 

An  early  set  of  experiments  tested  the  two  basic  assumptions  of  the  model  in 
a  more  conventional  experimental  paradigm. 

The  levels-of-abstraction  assumption  predicts  that  human  subjects  will  recog¬ 
nize  the  functional  differences  between  decisions  represent'ng  different  levels  of 
abstraction.  Accordingly,  in  one  experiment,  we  drew  statements  representing  diff¬ 
erent  levels  from  a  pool  of  thinking-aloud  protocols  and  asked  a  new  set  of  theoreti¬ 
cally  naive  subjects  to  sort  them  according  to  similarity.  We  then  analyzed  their 
sortings  of  the  statements,  using  a  hierarchical  clustering  program.  The  analysis 
produced  groups  of  statements  that  corresponded  exactly  to  the  levels  of  abstrac¬ 
tion  postulated  by  the  model.  Thus,  the  postulated  levels  represent  a  set  of  cognitive 
categories  for  planning  decisions. 

The  opportunism  assumption  predicts  that  a  planner’s  prior  decisions  should 
influence  the  generation  of  subsequent  decisions  in  both  top-down  and  bottom-up 
directions.  We  tested  two  aspects  of  this  prediction. 

One  experiment  assessed  the  impact  of  a  prior  decision  on  the  content  of  a 
subsequent  decision.  We  gave  subjects  planning  problems  that  required  them  to 
accept  a  particular  initial  decision  (e.g.,  to  go  to  the  drugstore  first).  Subjects  then 
chose  a  subsequent  decision  from  two  alternatives  (e.g..  to  travel  in  a  circle  around 
town  or  to  travel  from  west  to  east  across  town).  By  carefully  controlling  the  prior 
decision,  we  were  able  to  influence  subjects  to  choose  one  particular  option  from 
the  alternative  subsequent  decisions.  This  was  true  regardless  of  the  levels  of 
abstraction  represented  by  prior  and  subsequent  decisions. 

Another  experiment  assessed  the  impact  of  a  prior  decision  on  the  level  of 
abstraction  of  a  subsequent  decision.  Again  we  gave  subjects  planning  problems 
that  required  them  to  make  a  particular  initial  decision  and  to  choose  a  subsequent 
decision  from  two  alternatives.  In  this  case,  one  alternative  represented  a  higher 
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level  of  abstraction  than  the  prior  decision,  while  the  other  represented  a  lower 
level  of  abstraction.  By  carefully  controlling  the  prior  decision,  we  were  able  to 
influence  the  level  of  abstraction  of  the  chosen  subsequent  decision. 

These  results  supported  the  two  basic  assumptions  of  the  opportunistic  plan¬ 
ning  modei. 


GENERAL  PLANNING  STRATEGIES 

A  corollary  of  the  opportunism  assumption  is  that  planners  should  be  able  to 
adopt  alternative  general  planning  strategies.  In  particular,  they  should  be  able  to 
work  top-down,  developing  the  most  general  features  of  a  plan  before  working  out 
the  details,  or  bottom-up.  starting  immediately  with  the  details.  The  strategy  sub¬ 
jects  adopt  has  important  consequences  for  both  the  ease  of  generating  a  plan  and 
the  quality  of  the  plan  produced.  The  top-down  strategy  is  appropriate  when  time 
is  limited.  It  provides  the  most  efficient  approach  to  developing  a  plan  that  satisfies 
severe  time  limitations.  The  bottom-up  strategy  is  more  appropriate  when  there 
are  minimal  time  limitations.  Five  experiments  investigated  subjects’  abilities  and 
inclinations  to  adopt  these  alternative  strategies. 

Experiment  1  simply  assessed  subjects’  predispositions  to  adopt  each  strategy-. 
The  results  were  straightforward:  Most  subjects  adopted  the  bottom-up  strategy 
regardless  of  which  strategy  was  appropriate.  As  a  consequence  for  problems  that 
imposed  minimal  time  limitations,  subjects  produced  good  plans  in  a  short  period 
of  time.  However,  for  problems  that  imposed  time  limitations,  subjects  produced 
inferior  plans  and  required  longer  to  produce  them.  These  results  indicate  that 
subjects  have  strong  predilections  toward  adopting  the  bottom-up  strategy  rather 
than  the  top-down  strategy. 

Experiment  2  demonstrated  that  subjects  can  adopt  either  strategy  when  ex¬ 
plicitly  instructed  to  do  so.  It  further  documented  the  effect  of  the  adopted  strategy 
upon  performance.  When  instructed  to  adopt  the  top-down  strategy,  subjects  per¬ 
formed  better  on  time-limited  problems  than  on  unlimited  problems.  When  instruct¬ 
ed  to  adopt  the  bottom-up  strategy,  the  reverse  was  true. 

Experiment  3  illuminated  one  of  the  ways  in  which  planners  acquire  strategies, 
the  transfer  of  previously  successful  strategies  to  similar  problems.  We  instructed 
subjects  to  adopt  one  strategy  or  the  other  for  several  training  problems.  When 
given  a  new  problem  but  no  explicit  instructions,  subjects  continued  to  use  the 
strategy  they  used  on  the  training  problems.  They  transferred  the  old  strategy 
regardless  of  its  appropriateness,  with  the  expected  consequences  for  performance. 

Experiment  4  demonstrated  that  many  subjects  can  adopt  both  strategies  ap¬ 
propriately  if  they  are  explicitly  instructed  to  do  so.  However,  a  fair  number  of 
subjects  continued  to  show  a  predilection  toward  adopting  the  bottom-up  strategy. 

Experiment  5  showed  that  a  small  number  of  subjects  can  acquire  the  ability 
to  adopt  both  strategies  appropriately  on  the  basis  of  experience  alone.  We  instruct¬ 
ed  subjects  to  adopt  each  strategy  on  each  of  two  training  problems.  When  given 
s>  new  problem  but  no  explicit  instructions,  a  small  number  of  subjects  reliably 
transferred  the  appropriate  strategy.  However,  many  subjects  did  not  do  so  and 
instead  reverted  to  their  original  dependence  upon  the  bottom-up  strategy. 

These  results  confirm  the  opportunistic  model's  prediction  that  subjects  can  use 
alternative  general  planning  strategies  and  demonstrate  the  important  conse- 


quences  these  strategies  have  for  planning  performance.  However,  the  results  also 
show  a  predilection  among  most  subjects  to  adopt  a  particular  strategy  and  a 
reluctance  among  some  subjects  to  alter  this  predilection. 


INDIVIDUAL  DIFFERENCES  IN  PLANNING 

The  opportunistic  planning  model  embodies  three  potential  dimensions  for 
individual  differences  in  planning  behavior.  First,  individuals  may  differ  in  their 
operational  planning  spaces.  Better  planners  may  generate  more  decisions  on  cer¬ 
tain  planes  of  the  blackboard  or  at  particular  levels  of  abstraction.  Second,  individ¬ 
uals  may  differ  in  their  control  processes.  Better  planners  may  show  greater  flexi¬ 
bility  in  their  allocation  of  attention  among  different  aspects  of  the  planning  prob¬ 
lem.  Third,  individuals  may  differ  in  planning  knowledge.  Better  planners  may 
have  more  planning  heuristics  or  more  powerful  planning  heuristics. 

We  investigated  these  hypotheses  by  examining  in  detail  a  number  of  thinking- 
aloud  protocols.  We  examined  thirty  protocols  in  all,  six  from  each  of  five  subjects. 
We  evaluated  each  protocol  with  a  composite  score  based  on  number  of  planned 
errands,  importance  of  planned  errands,  satisfaction  of  temporal  constraints  spe¬ 
cified  in  the  problem,  route  efficiency,  and  temporal  realism.  Averaging  scores  for 
the  si:,  protocols  from  each  subject,  we  generated  a  "planning  score”  for  each 
subject.  Subjects  with  high  scores  were  identified  as  good  planners;  those  with  low 
scores  were  identified  as  poor  planners.  We  then  examined  the  relationship  be¬ 
tween  each  of  the  dimensions  described  above,  as  manifested  in  the  protocols,  and 
the  subjects’  planning  scores. 

We  found  that  good  planners  make  more  metaplan  and  executive  decisions 
than  poor  planners,  suggesting  that  they  exercise  more  conscious  control  over  the 
planning  process.  Good  planners  also  make  more  world  knowledge  decisions,  sug¬ 
gesting  that  they  are  more  sensitive  to  available  information  about  the  plan  execu¬ 
tion  environment.  We  also  found  that  good  planners  make  more  decisions  at  high 
levels  of  abstraction.  This  is  consistent  with  our  findings  on  general  planning  strate¬ 
gies,  suggesting  that  good  planners  recognize  the  importance  of  a  global  approach 
to  some  planning  problems,  while  poor  planners  rely  more  heavily  on  purely  bot¬ 
tom-up  planning  strategies. 

We  found  that  good  planners  exhibit  greater  attentional  flexibility  than  poor 
planners.  They  more  frequently  shift  their  focus  of  attention  among  the  different 
planes  of  the  planning  blackboard  and  among  different  temporal  loci  within  the 
plan  itself. 

Finally,  we  found  that  good  planners  have  more  planning  heuristics  as  well  as 
more  powerful  planning  1  'ristics  than  poor  planners.  This  was  true  for  heuristics 
applying  on  each  of  the  five  planes  of  the  planning  blackboard. 

Based  on  these  results,  we  developed  a  preliminary  protocol  for  effective  plan¬ 
ning  in  this  domain: 

1.  Identify  the  most  important  errands. 

2.  Identify  spatial  clusters  of  errands 

3.  Assess  constraints  on  the  plan. 

4.  Establish  evaluation  criteria. 
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5.  Choose  a  subset  of  the  errands  as  plan  objectives,  based  on  importance, 
location,  time  limitations,  constraints,  etc. 

6.  Anchor  the  plan  with  temporally  constrained  objectives. 

7.  Establish  a  general  design  for  the  plan. 

8.  Plan  forward  and  backward  from  anchored  objectives  in  accordance  with 
the  design. 

9.  Evaluate  the  plan  against  established  criteria  throughout  the  process. 


EFFECTS  OF  SCENARIO  CHARACTERISTICS 

Scenario  characteristics  can  have  important  effects  on  the  planning  process  and 
the  plan  produced.  The  opportunistic  planning  model  suggests  that  certain  char¬ 
acteristics  should  interact  with  particular  planning  heuristics  (e.g.,  the  interaction 
of  time  limitations  with  general  planning  strategy).  However,  there  may  be  other 
scenario  characteristics  that  have  more  general  effects  on  planning  performance. 
Accordingly,  we  conducted  a  correlational  study  relating  a  variety  of  scenario 
characteristics  to  several  measures  of  planning  performance.  The  results  suggested 
the  following  relationships: 

1.  Planning  time  increases  with  the  number  of  goals  considered  and  with  the 
time  available  for  achieving  goals.  This  relationship  reflects  the  increasing 
complexity  of  the  planning  task  as  number  of  goals  and  execution  time 
increase. 

2.  Subjects  are  uniformly  unrealistic.  They  consistently  overestimate  what 
they  can  accomplish  in  the  time  available  to  execute  their  plans.  Further, 
as  the  difficulty  of  achiev;  g  all  of  the  goals  in  the  available  time  increases, 
subjects  become  increasingly  unrealistic. 

3.  As  the  difficulty  of  achieving  all  of  the  goals  in  the  available  time  in¬ 
creases,  subjects  plan  to  achieve  fewer  goals  and  less  important  goals. 
Thus,  they  sacrifice  both  quantity  and  quality  as  the  task  becomes  more 
difficult. 


REALISM  IN  PLANNING 

One  of  the  most  provocative  findings  revealed  by  the  correlational  study  sum¬ 
marized  above  was  subjects’  tendency  to  overestimate  what  they  could  accomplish 
in  the  time  available  for  plan  execution.  We  conducted  several  additional  experi¬ 
ments  to  investigate  this  phenomenon  further. 

The  first  experiment  replicated  the  original  results  in  an  experimental  para¬ 
digm.  This  experiment  confirmed  subjects’  tendency  to  overestimate  what  they  can 
accomplish  and  showed  that  the  tendency  increases  as  the  number  of  goals  con¬ 
sidered  increases  or  as  the  time  available  for  achieving  goals  decreases.  It  also 
suggested  that  subjects  are  not  entirely  insensitive  to  time  limitations.  As  time 
stress  increased  in  either  of  these  two  ways,  subjects  responded  appropriately  by 
planning  a  smaller  proportion  of  the  goals  under  consideration.  However,  they 
failed  co  reduce  their  aspirations  sufficiently  to  satisfy  specified  time  limitations. 

We  generated  two  hypotheses  to  explain  subjects’  unrealistic  planning  behav¬ 
ior.  The  cognitive  hypothesis  assumes  that  people  tend  to  estimate  time  require- 


merits  at  high  levels  of  abstraction.  In  so  doing,  they  fail  to  enumerate  all  of  the 
time-consuming  details  of  planned  actions,  and  as  a  consequence,  they  consistently 
underestimate  the  time  required  to  perform  them.  The  motivational  hypothesis 
assumes  that  people  are  biased  by  their  aspirations.  They  wish  to  achieve  all  of  the 
goals  under  consideration  and  this  influences  their  estimates  of  required  times.  The 
two  hypotheses  are  not  mutually  exclusive;  both  may  contribute  to  the  phenome¬ 
non. 

To  test  the  cognitive  hypothesis,  we  required  subjects  to  estimate  time  require¬ 
ments  at  three  different  levels  of  abstraction.  At  the  most  general  level,  they  gave 
quick  estimates  of  the  time  required  to  achieve  a  set  of  goals.  At  an  intermediate 
level,  they  estimated  the  times  required  to  achieve  each  goal  in  a  set  of  goals.  At 
the  more  specific  level,  they  estimated  the  time  required  to  perform  the  component 
actions  entailed  in  each  goal  in  a  set  of  goals.  The  results  confirmed  the  hypothesis; 
subjects  produced  longer  time  estimates  at  lower  levels  of  abstraction  than  at 
higher  levels. 

To  test  the  motivational  hypothesis,  we  gave  subjects  two  types  of  planning 
problems,  differing  only  in  the  importance  of  the  goals  under  consideration.  We 
reasoned  that  problems  specifying  very  important  goals  should  produce  high 
motivational  states,  while  those  specifying  moderately  important  goals  should  pro¬ 
duce  lower  motivational  states.  Under  the  motivational  hypothesis,  subjects  should 
plan  more  unrealistically  as  the  importance  of  the  goal  is  increased.  The  results 
confirmed  this  prediction.  Subjects’  plans  were  more  unrealistic  for  problems  speci¬ 
fying  very  important  goals  than  for  problems  specifying  moderately  important 
goals. 

These  results  suggest  that  subjects’  tendency  to  plan  more  than  they  can  accom¬ 
plish  in  the  available  time  reflects  both  cognitive  and  motivational  factors. 


PLAN  EVALUATOR 

It  is  often  difficult  to  assess  planning  performance  because  of  the  inherent 
ambiguity  in  defining  a  good  plan.  One  approach  is  to  evaluate  the  performance  of 
a  plan.  A  plan  that  maximizes  the  goals  achieved  and  minimizes  the  resources 
consumed  is  a  superior  plan.  The  problem  is  that  plans  sometimes  succeed  or  fail 
for  reasons  that  have  little  to  do  with  the  quality  of  the  plan  itself.  For  example, 
low-probability  events  in  the  environment  may  undermine  a  plan  that  would  other¬ 
wise  outperform  alternative  plans. 

Another  approach  is  to  evaluate  a  plan  against  various  criteria.  The  problem 
with  this  approach  is  that  there  are  many  possible  criteria,  and  individuals  may 
differ  in  the  significance  they  attach  to  particular  ones.  In  addition,  some  criteria 
may  conflict  with  one  another.  For  example,  the  planner  can  try  to  formulate  a  plan 
that  includes  many  goals  or  a  plan  that  includes  the  most  important  goals.  It  is  not 
always  possible  to  satisfy  both  criteria  with  a  single  plan.  Individuals  may  differ 
in  their  judgments  about  which  criteria  to  compromise  in  such  situations 

Our  approach  to  the  assessment  of  planning  performance  was  to  create  a  simu¬ 
lated  planning  environment  with  which  subjects  could  interact.  Subjects  formulate 
plans  and  submit  them  to  a  computer  program  which  then  simulates  plan  execu¬ 
tion.  The  program  provides  feedback  regarding  the  plan’s  outcomes  and  the  vari- 
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ous  attributes  of  the  plan  that  contributed  to  its  outcomes.  For  example,  the  subject 
might  learn  that  execution  time  ran  out  before  all  of  the  goals  were  achieved  and 
that  this  happened  because  his  time  estimates  were  unrealistic.  Based  on  these 
attributes,  the  program  also  computed  and  reported  an  overall  plan  score.  We 
allowed  subjects  to  interact  with  the  simulated  environment  on  several  trials.  In 
the  course  of  these  interactions,  subjects  learned  about  the  environmental  contin¬ 
gencies  that  might  impact  on  their  plans  and  about  the  criteria  that  would  be  used 
to  evaluate  their  plans.  We  evaluated  subjects  in  terms  of  their  asymptotic  plan 
scores.  Thus,  our  evaluation  of  their  performance  was  not  confused  by  surprising 
environmental  contingencies  or  by  individual  differences  in  planning  criteria.  It 
was  a  relatively  pure  measure  of  the  subject’s  ability  to  plan  in  accordance  with  a 
known  set  of  contingencies  and  criteria. 


VI.  PRELIMINARY  PRINCIPLES  FOR  IMPROVING 
PLANNING  PERFORMANCE 


The  body  of  research  summarized  in  this  Report  suggests  a  number  of  prin¬ 
ciples  for  improving  planning  performance,  as  outlined  below 


CRITERIA  FOR  SELECTING  PLANNERS 

1.  Large-capacity  working  memory.  The  opportunistic  model  describes  peo¬ 
ple’s  tendency  to  "jump  around’’  the  space  of  possible  considerations  while  forming 
plans.  This  suggests  that  it  may  be  important  for  planners  to  have  large-capacitv 
working  memories  in  order  to  keep  track  of  several  aspects  of  a  developing  plan 
simultaneously. 

2.  Attentional  flexibility.  Our  studies  of  individual  differences  in  planning 
performance  showed  that  good  planners  shift  attention  among  different  aspects  of 
a  planning  problem  more  frequently  than  poor  planners.  Therefore,  attentional 
flexibility  may  be  another  important  characteristic  of  potential  planners. 

3.  Strategic  flexibility.  Our  studies  of  top-down  versus  bottom-up  planning 
strategies  showed  the  impact  of  particular  planning  strategies  on  the  efficacy  of  the 
plans  subjects  produce  and  on  the  efficiency  with  which  they  produce  them.  In 
addition,  some  subjects  appeared  more  willing  than  others  to  adopt  alternative 
strategies.  Therefore,  strategic  flexibility  may  be  another  important  selection  crite¬ 
rion. 


WHAT  TO  TEACH  PLANNERS 

4.  Concepts  of  abstract  plans,  world  knowledge  decisions,  metacognitive  deci¬ 
sions,  and  executive  decisions.  Our  studies  of  individual  differences  showed  that 
good  planners  made  decisions  in  all  categories  of  the  planning  blackboard,  whereas 
poor  planners  made  only  certain  kinds  of  decisions.  In  particular,  high-level  ab¬ 
stract  decisions,  world  knowledge  decisions,  metacognitive  decisions,  and  executive 
decisions  distinguished  good  planners  from  poor  planners.  Therefore,  planners 
should  be  taught  the  roles  of  these  types  of  decisions  in  the  planning  process. 

5.  Domain-specific  planning  heuristics.  Our  studies  of  individual  differences 
also  showed  that  good  planners  had  more  different  planning  specialists  than  poor 
planners.  Therefore,  planners  should  be  taught  a  variety  of  domain-specific  plan¬ 
ning  heuristics. 

6.  Costs  and  benefits  of  opportunism.  There  is  considerable  evidence  that  most 
people  employ  some  amount  of  opportunism  in  the  planning  process.  Opportunism 
provides  planners  freedom  to  examine  various  aspects  of  a  problem,  investigate 
alternative  plan  configurations,  etc.  This  enables  them  to  discover  solutions  that 
more  rigid  approaches  would  obscure.  On  the  other  hand,  opportunism  requires 
additional  time  and  may  lead  planners  down  unproductive,  as  well  as  productive, 
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solution  paths.  Planners  should  be  taught  these  advantages  and  disadvantages  and 
how  to  exercise  controlled  opportunism. 

7.  General  planning  strategies.  As  mentioned  above,  different  planning  strate¬ 
gies  are  appropriate  under  different  circumstances.  Planners  should  be  taught 
general  planning  strategies  and  the  circumstances  under  which  each  is  appropri¬ 
ate. 

8.  Judgment  and  time  estimation.  Most  people  show  a  strong  tendency  to 
underestimate  the  time  required  for  planned  actions.  As  a  consequence,  their  pians 
are  unrealistic  and  overrun  the  time  available  for  execution.  Planners  should  be 
taught  cognitive  methods  for  making  such  judgments  more  reliably  and  more 
accurately. 


HOW  TO  TRAIN  PLANNERS 

9.  Provide  explicit  instruction.  Explicit  instruction  appears  to  be  a  highly 
effective  technique  for  training  particular  planning  strategies  and  methods. 

10.  Induce  illustrative  experiences.  Many  planners  seem  to  be  able  to  general¬ 
ize  what  they  learn  from  one  planning  problem  to  subsequent,  similar  planning 
problems.  Therefore,  general  strategies  and  methods  can  be  taught  by  instructing 
planners  how  to  use  them  on  specific  problems  and  providing  opportunities  for 
them  to  generalize  them  to  similar  problems. 

11.  Illustrate  effective  planning  with  a  simulation.  Because  our  planning 
simulation  effectively  mimics  the  cognitive  processes  people  use  while  planning,  it 
may  provide  a  useful  model  during  training.  The  simulation  could  be  programmed 
to  illustrate  alternative  planning  strategies. 


EFFECTIVE  AIDS  FOR  PLANNERS 

12.  Aids  for  reliable  estimation  of  resource  requirements.  Even  with  training, 
people  may  not  be  able  to  generate  reliable  and  accurate  estimates  of  resource 
requirements.  However,  it  would  be  fairly  straightforward  to  develop  computer 
aids  for  generating  these  estimates. 

13.  Plan-evaluation  aids.  People  appear  to  be  better  at  generating  plans  than 
they  are  at  evaluating  them.  It  would  be  useful  to  have  aids  that  evaluated  a  plan’s 
feasibility  and  efficiency  against  some  model  of  the  plan  execution  environment. 

14.  A  planning  collaborator.  Planning  is  a  complex  problem.  Even  the  sophis¬ 
ticated  cognitive  skills  people  possess  do  not  always  produce  optimal  or  even  satis¬ 
factory  plans.  It  would  be  useful  to  have  a  planning  collaborator  that  suggested 
useful  planning  strategies  and  heuristics  and  contributed  specific  decisions  to  the 
plan  in  progress. 

While  these  principles  must  be  regarded  as  preliminary,  they  follow  from  a 
substantial  and  integrated  body  of  research.  We  believe  that  they  provide  a  firm 
foundation  for  future  research  on  planning  and  for  the  development  of  improved 
planning  technologies. 
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